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BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an image 
processing apparatus, an image processing method and a 
computer-readable recording medium or the like, and, more 
particularly, to an image processing apparatus, an image 
processing method and a recording medium which are suitable 
for application to an electronic camera or the like. 

Description of Related Art 

Fig. 1 is a block diagram schematically showing 
the construction of a general digital electronic camera. In 
the digital electronic camera shown in Fig. 1, a CPU 102 
(made of a microprocessor) serves to control the entire 
system of the digital electronic camera. A DRAM 103 is a 
memory element called a dynamic RAM (hereinafter referred to 
as the DRAM) into or from which to write or read image data, 
camera system programs, work data for the CPU 102 and the 
like. The DRAM 103 needs to perform an operation called 
refresh by a predetermined number of times for a 
predetermined time period so that the DRAM 103 can retain 
the contents stored therein. A block 104 is a DRAM access 
circuit for converting the signal outputted from the CPU 102 
into a DRAM control signal. A refresh signal generating 
circuit 106 generates a refresh signal by producing a pulse 
to be outputted to a RAS (row address strobe) clock line and 
CAS (column address strobe) clock line which constitute DRAM 
control lines . 
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An image pickup circuit 107 is a circuit for 
converting an image picked up by a CCD image pickup element 
(not shown) into digital image data and recording the 
digital image data on the DRAM 103. An image display 
circuit 108 is a circuit for transferring image data from 
the dram 103 to a liquid crystal display panel, or 
converting the image data into a video signal and outputting 
the video signal to an external video output. An image 
compressing/expanding circuit 109 is a circuit for 
compressing image data to reduce the amount of information 
thereof or expanding compressed image data into data having 
the amount of information that is equivalent to the amount 
of information of the original image data. A vertical 
addition mixing circuit 110 is a circuit for adding 
vertically adjacent lines together to enable video 
processing of image data for two fields which are outputted 
from an interlaced CCD image pickup element. An image 
processing circuit 111 is a circuit for performing the 
processing of changing a size including the aspect ratio of 
image data on the DRAM 103, rotating an image, or changing 
the position of an image in the DRAM 103. A DRAM bus 113 
includes a data bus, a DRAM address bus and the DRAM control 
lines . 

The operation of the digital electronic camera 
shown in Fig. 1 will be described below. 

First of all, the image pickup operation of the 
digital electronic camera will be described below. Image 
data formed on the CCD image pickup element is read from the 
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image pickup circuit 107 and is temporarily recorded in the 
DRAM 103. Then, the vertical addition mixing circuit 110 
adds together vertically adjacent lines of the image data on 
the DRAM 103, and again records the result in the DRAM 103. 
Then, the image pickup circuit 107 converts the added image 
data stored in the DRAM 103 into luminance color-difference 
image data, and again records the result in the DRAM 103. 
The image processing circuit 111 optimizes the aspect ratio 
of the luminance color-difference image data. Finally, the 
image data is compressed by the image compressing/expanding 
circuit 109, and the picked-up image is recorded as a file 
on a recording medium (not shown) via the DRAM bus 113. 

The image display operation of the digital 
electronic camera will be described below. First of all, an 
image file is read from the recording medium by the image 
compressing/expanding circuit 109, and is expanded into 
image data in the DRAM 103. The image display circuit 108 
reads the image data from the DRAM 103, and outputs the 
image data to a monitor (not shown). Thus, the image 
display operation is achieved. 

To normally effect the above-described image 
pickup and display processing, it is necessary that the DRAM 
103 be correctly refreshed in the aforesaid processing cycle 
and that use rights to the data bus or the DRAM 103 be 
correctly assigned to each image processing, refresh and the 
CPU 102. 

The DRAM 103 needs to perform an operation called 
refresh by not less than a predetermined number of times for 
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a predetermined time period. It is assumed herein that the 
DRAM 103 needs 1,024 refreshes per 128 msec (milliseconds). 

However, the image pickup circuit 107 and the 
image display circuit 108 operate at horizontal and vertical 
periods. If refresh is performed during an image pickup 
operation, the image pickup operation is interrupted during 
the refresh. This leads to the problem that a picked-up 
image partly lacks data and no correct image can be 
obtained . 

To avoid this problem, the refresh of the DRAM 103 
is performed during the blanking periods of an image signal 
in each of an image pickup operation and an image display ■ 
operation. The image pickup circuit 107 outputs a 
horizontal synchronizing signal. Several-microsecond 
periods before and after the horizontal synchronizing signal 
are horizontal blanking periods, during which data read from 
the CCD image pickup element do not appear. During such a 
horizontal blanking period, the DRAM 103 is refreshed 
because the image pickup circuit 107 does not need to record 
data in the DRAM 103. 

When the horizontal synchronizing signal is "1", 
the refresh signal generating circuit 106 generates a 
refresh signal of one cycle and outputs this refresh signal 
to both the RAS and CAS clock lines of the DRAM bus 113. it 
is assumed here that the period of the horizontal 
synchronizing signal outputted from the image pickup circuit 
107 is 85 jusec (microseconds). if one refresh is performed 
each time one horizontal synchronizing signal is outputted, 



-5- 



1,506 refreshes per 128 msec are performed. This satisfies 
the prescribed condition of the DRAM 103, i.e., 1,024 
refreshes per 128 msec. 

The horizontal synchronizing signal is coupled to 
the hold request input of the CPU 102. While the CPU 102 is 
receiving the horizontal synchronizing signal, the CPU 102 
temporarily stops its operation to avoid access to the DRAM 
103 during refresh. 

The image display circuit 108 operates in 
synchronism with clocks and horizontal synchronizing signals 
outputted from the image pickup circuit 107. The image 
display circuit 108 starts reading display image data 
several microseconds after the reception of a horizontal 
synchronizing signal. Assuming that the number of 
horizontal data of a picked-up image is equal to the number 
of horizontal data of a display image, the display of one 
horizontal line is completed before the next horizontal 
synchronizing signal appears. Since refresh during an image 
display operation is performed in synchronism with the 
horizontal synchronizing signal similarly to refresh during 
an image pickup operation, the refresh during the image 
display operation is performed during the horizontal 
blanking periods thereof. 

When the image compressing /expanding circuit 109 
receives a compression command, the image 

compressing/expanding circuit 109 compresses the data stored 
in the DRAM 103 into a predetermined size. The time period 
required for such compression processing ranges from several 
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hundred milliseconds to several seconds according to image 
sizes, it is, therefore, necessary to interrupt the 
compression processing and refresh the DRAM 103 at intervals 
of a predetermined time period. 

When the horizontal synchronizing signal becomes 
"1", the refresh signal generating circuit 106 causes the 
DRAM 103 to start refreshing and outputs a signal indicating 
that refresh is being performed. When the image 
compressing/expanding circuit 109 receives this signal, the 
image compressing/expanding circuit 109 interrupts its 
processing. During this time, the refresh signal generating 
circuit 106 outputs a refresh signal to the DRAM 103. If 
the horizontal synchronizing signal again becomes "0", the 
signal indicating that refresh is being performed again 
becomes "0" and the image compressing/expanding circuit 109 
restarts the compression processing. 

Similarly to the image compressing/expanding 
circuit 109, the vertical addition mixing circuit 110, the 
image processing circuit 111 and other image processing 
circuits operate while interrupting their processing during 
refresh. 

However, the above-described general digital 
electronic camera has the following problems. 

During the operation of a process such as the 
above-described image processing, the CPU 102 is held and 
temporarily stops its operation, so that the CPU 102 is not 
able to perform any operation other than the process. For 
example, during image compression, the CPU 102 is not able 
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to perform the processing of displaying characters such as 
"COMPRESSING" on a liquid crystal display panel or the 
processing of blinking a system busy indicator lamp which 
indicates that it is impossible to start an image pickup 
operation . 

In addition, since the unit time of refresh is one 
horizontal blanking period, refresh is performed by the 
number of times which is greater than the actually necessary 
number of times. in the aforesaid example, 1,506 refreshes 
per 128 msec are performed, which are approximately 1.5 
times as large as the actually necessary number of refreshes 
per unit time, i.e., 1,024 refreshes per 128 msec. As a 
result, power consumption increases. Particularly during 
sleep, since the consumption of power due to refresh 
occupies a large proportion of the energy consumption of the 
entire camera, the above-described fact is not preferable. 

Furthermore, since each of the image 
compressing/expanding circuit 109, the vertical addition 
mixing circuit 110 and the image processing circuit 111 
receives refresh requests by a large number of times, the 
frequency with which its process is interrupted becomes 
higher, and the processing time period of the process 
becomes longer. This impairs the operability of the digital 
electronic camera. 

During sleep or image display, since no image 
pickup operation needs to be performed, it is preferable to 
stop the supply of power to the image pickup circuit 107. 
However, in the above -described example, since refresh 

-8- 



timing is generated on the basis of the operation of the 
image pickup circuit 107, it is impossible to stop the image 
pickup circuit 107 in any situation. Accordingly, it is 
impossible to achieve a sufficient reduction in power 
consumption . 

BRIEF SUMMARY OF THE INVENTION 

It is, therefore, an object of the present 
invention to solve the above-described problems. 

Another object of the present invention is to 
provide an image processing apparatus and method as well as 
a computer-readable recording medium all of which are 
capable of reliably executing various kinds of image 
processing while reliably performing a refresh operation. 

To achieve the above object, in accordance with 
one aspect of the present invention, during the execution of 
a refresh process for storage means which stores image data, 
a first process using the storage means and a second process 
using the storage means, access to the storage element is 
arbitrated so that the first process takes priority over the 
refresh process and the refresh process takes priority over 
the second process. 

Another object of the present invention is to 
provide an image processing apparatus and method as well as 
a computer-readable recording medium all of which are 
capable of reducing the refresh rate of storage means to the 
minimum necessary rate to reduce power consumption and to 
enhance the speeds of other processes. 
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To achieve the above object, in accordance with 
another aspect of the present invention, during the 
execution of a refresh process for refreshing storage means 
which stores image data and a process for processing the 
image data by performing writing of the image data into the 
storage means or reading of the image data from the storage 
means, whether to synchronize the refresh process with an 
image is specified according to the contents of the image 
data processing process. 

Another object of the present invention is to 
provide an image processing apparatus and method as well as 
a computer-readable recording medium all of which enable a 
CPU or the like to efficiently execute image processing as 
well as processing other than refresh. 

To achieve the above object, in accordance with 
another aspect of the present invention, during switching 
from a refresh process for refreshing storage means which 
store image data to a process for processing image data by 
performing writing of the image data into the storage means 
or reading of the image data from the storage means, another 
operation is controlled by using the storage means. 

The above and other objects, features and 
advantages of the present invention will become apparent 
from the following detailed description of preferred 
embodiments of the present invention, taken in conjunction 
with the accompanying drawings. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
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Fig. 1 is a block diagram schematically showing 
the construction of a general digital electronic camera . ; 

Fig. 2 is a block diagram schematically showing 
the construction of a digital electronic camera according to 
one embodiment of the present invention; 

Fig. 3 is a block diagram showing the essential 
portion of a first type of process circuit incorporated in 
the electronic camera shown in Fig. 2; 

Fig. 4 is a block diagram showing the essential 
portion of a second type of process circuit incorporated in 
the electronic camera shown in Fig. 2; and 

Fig. 5 is a diagram showing the state transition 
of a bus arbitrating circuit in the electronic camera shown 
in Fig . 2 . 



DETAILED DESCRIPTION OF THE INVENTION 

A preferred embodiment of the present invention 
will be described below in detail with reference to the 
accompanying drawings . 

Fig. 2 diagrammatically shows the construction of 
a digital electronic camera according to the preferred 
embodiment of the present invention. 

The digital electronic camera shown in Fig. 2 
includes a DRAM bus arbitrating circuit 101 for arbitrating 
accesses to a DRAM bus by a CPU 102, a process circuit and a 
refresh circuit both of which will be described later, a 
refresh timing counter 105 for generating one pulse at 
intervals of a predetermined time period, OR circuits 112, 
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115 and 116, a refresh holding counter 114 , the CPU 102 and 
other constituent elements 103, 104, 106 to 111 and 113. 
Since the constituent elements 102 to 104, 106 to 111 and 
113 have been previously described with reference to Fig. 1, 
the description thereof is omitted for the sake of 
simplicity. The DRAM 103, in which a program for the CPU 
102 is also stored, constitutes a computer-readable storage 
medium according to the present invention. 

The operation of the digital electronic camera 
according to the preferred embodiment of the present 
invention will be described below. The digital electronic 
camera is arranged to execute processes including an image 
pickup process, an image display process, an image 
compressing /expanding process, a vertical addition mixing 
process and an image processing process. First of all, the 
operations of the respective processes and the operation of 
the DRAM bus arbitrating circuit 101 will be described 
below. Each of the process circuits 107 to 111 is 
classified into a process circuit of the type A shown in 
Fig. 3 or a process circuit of the type B shown in Fig. 4. 

Referring to the process circuit of the type A 
shown in Fig. 3, when a command processing circuit 301 
receives a process start command from the CPU 102, the 
command processing circuit 301 generates a process request 
signal P_REQ indicative of "1". This process request signal 
P_REQ passes through the OR circuit 112 and is inputted into 
the dram bus arbitrating circuit 101. When the DRAM bus 
arbitrating circuit 101 receives the process request signal 
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P_REQ indicative of "1", the DRAM bus arbitrating circuit 
101 requests a refresh or a CPU access to release an access 
right to the DRAM bus 113. If neither the refresh nor the 
CPU access has occurred, the DRAM bus arbitrating circuit 
101 directly proceeds to the next step. Although not show 
in Fig. 2, a connection line for transmission or reception 
of commands is disposed between the CPU 102 and each of the 
circuits 107 to 111. 

If the DRAM bus 113 is released, the DRAM bus 
arbitrating circuit 101 gives a DRAM bus use right to a 
process circuit 302 and then generates a signal P_ACK 
indicative of "1" which means that the DRAM bus use right is 
given to the process circuit 302. When the process circuit 
302 receives the signal P_ACK indicative of "1", the process 
circuit 302 starts a predetermined process. Upon the 
completion of the predetermined process, the process circuit 
302 resets the process request signal P_REQ to "0". When 
the DRAM bus arbitrating circuit 101 receives the process 
request signal P_REQ indicative of "0", the DRAM bus 
arbitrating circuit 101 returns to a standby state and waits 
until a DRAM access, a refresh or a process by the CPU 102 
again requests a DRAM bus use right. 

The process circuit of this type A, once it starts 
a process, executes the process in preference to any request 
from another circuit. Each of the image pickup circuit 107 
and the image display circuit 108 is realized as the process 
circuit of the type A. When an image pickup operation or an 
image display operation is to be started, the corresponding 
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one of the image pickup circuit 107 and the image display 
circuit 108 outputs a process request signal P_REQ 
indicative of "1" and acquires an access right to the DRAM 
bus 113. Upon the completion of the operation of picking up 
or displaying an image for one line, the corresponding 
circuit 107 or 108 resets the process request signal p_req 
to "0" and releases the DRAM bus 113. 

Thus, a refresh operation is prevented from being 
started during the operation of picking up or displaying an 
image for one line, whereby an image can be picked up or 
displayed without interruption. A refresh is performed 
after the release of the DRAM bus 113, i.e., during a 
horizontal blanking period. The refresh operation will be 
described later. 

The process circuit of the type B shown in Fig. 4 
will be described below. When a command processing circuit 
401 receives a process start command from the CPU 102, the 
command processing circuit 401 outputs a signal of "0". In 
the meantime, when a timer 403 receives a signal VR_REQ 
indicative of "1", the timer 4 03 is reset and starts 
counting a predetermined time period. While the timer 4 03 
is performing a counting operation, the timer 403 outputs a 
signal of "0"; otherwise, the timer 403 outputs a signal of 
"1". The output of the command processing circuit 401 and 
the output of the timer 403 are ANDed by an AND circuit 404, 
and the obtained logical product is output ted as a process 
request signal P_REQ. Thus, the process circuit of the type 
B, when it receives a start command, outputs the process 
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request signal P_REQ indicative of "1", but if the process 
circuit receives a signal of "1" at the terminal VR_REQ (to 
be described later) which interlocks with a refresh request, 
the process circuit of the type B outputs a process request 
signal p_req indicative of "0" during a predetermined time 
period which corresponds to a value written into a time 
setting register 405 by the CPU 102. 

So long as the output of the timer 403 is not "0", 
when the command processing circuit 401 receives a process 
start command from the CPU 102, the process circuit of this 
type B generates a process request signal P_REQ indicative 
of "1". This signal P_REQ passes through the OR circuit 112 
and is inputted into the DRAM bus arbitrating circuit 101. 
When the DRAM bus arbitrating circuit 101 receives the 
process request signal P_REQ indicative of "1", the DRAM bus 
arbitrating circuit 101 requests a refresh or a CPU access 
to release an access right to the DRAM bus 113. If neither 
the refresh nor the CPU access has occurred, the DRAM bus 
arbitrating circuit 101 directly proceeds to the next step. 

If the DRAM bus 113 is released, the DRAM bus 
arbitrating circuit 101 gives a DRAM bus use right to a 
process circuit 402 and then generates a signal P_ACK 
indicative of "1" which means that the DRAM bus use right is 
given to the process circuit 402. When the refresh timing 
counter 105 which will be described later outputs a signal 
of "1" to the refresh holding counter 114, the refresh 
holding counter 114 receive this signal indicative of "1" 
and outputs a signal R_REQ indicative of "1". As will be 
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described later, when the refresh timing counter 105 outputs 
the signal of "1", the timer 403 is reset, and starts a 
counting operation and outputs a signal of "0". 
Accordingly, the process request signal P_REQ is reset to 
"0", and the process circuit 4 02 temporarily stops. 

When the DRAM bus arbitrating circuit 101 
determines that the signal R_REQ is "1" and the signal P_REQ 
is "0", the DRAM bus arbitrating circuit 101 gives the DRAM 
bus use right to refresh processing and then outputs a 
refresh instruction signal to the refresh signal generating 
circuit 106. The refresh signal generating circuit 106 
outputs a refresh signal to the DRAM bus 113 and refreshes 
the DRAM 103. 

Even after the refresh of the DRAM 103 has been 
completed, the timer 403 continues to output a signal of "0" 
until a predetermined time period elapses. Accordingly, the 
process request signal P_REQ remains "0". During this time, 
the DRAM bus 113 remains released. Accordingly, the CPU 102 
is allowed to operate while using the DRAM bus 113. 

When the timer 403 counts up the predetermined 
time period, the process request signal P_REQ is again set 
to "1". When the process circuit receives a signal P_ACK 
indicative of "1", the process circuit restarts its process. 
If the process circuit completes its process operation, the 
command processing circuit 401 outputs a signal of "0" so 
that the process request signal P_REQ is reset to "0". 

Each of the image compressing/expanding circuit 
109, the vertical addition mixing circuit 110 and the image 
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processing circuit 111 has a comparatively long processing 
time period and can be arranged so that even if its 
processing is interrupted at an intermediate step, the 
result of the processing is not affected by such 
interruption. For this reason , each of the circuits 109, 
110 and 111 is arranged as the process circuit of the type 
B. The set value of the time setting register 405 in each 
of the circuits 109, 110 and 111 will be described later. 

The refresh operation will be described below. By 
way of example, it is assumed that the DRAM 103 used in the 
present embodiment needs to be refreshed by 1,024 times for 
128 msec. Referring to Fig. 2, the refresh timing counter 
105 generates one pulse at intervals of 125 /jsec. The 
refresh holding counter 114 is an up/down counter capable of 
counting up to 16. When the refresh holding counter 114 
receives one pulse from the refresh timing counter 105, the 
refresh holding counter 114 increments its count value by 
one, and if the DRAM 103 is refreshed once, the refresh 
holding counter 114 decrements the count value by one. When 
the count value is not less than "1", the refresh holding 
counter 114 outputs a signal R_REQ indicative of "1". 

In addition, the DRAM bus arbitrating circuit 101 
is arranged to output a signal R_ACK indicative of "1" to 
the refresh signal generating circuit 106 when the count 
value of the refresh holding counter 114 is not less than 
"1" and the DRAM bus 113 is not occupied by another process. 
The operation of outputting the signal R_ACK indicative of 
"1" is repeated until the count value of the refresh holding 
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counter 114 reaches "0". 

Thus, such refresh is performed once per 125 jusec, 
but even if the DRAM bus 113 is occupied for a time period 
longer than 125 jisec by the process circuit of the type A, 
the refresh is performed once per 125 ^sec as long as the 
time period is shorter than 2,000 ^sec (= 125 pisec x 16). 
This time period is sufficiently long if one horizontal 
synchronizing period of an image pickup or display operation 
is 85 psec. Thus, the prescribed condition of the DRAM 103, 
i.e., 1,024 refreshes per 128 msec, is satisfied, and the 
system of the digital electronic camera can operate stably. 

Assuming that the DRAM 103 needs to be refreshed 
by 1,024 time for 16.4 msec, the pulse generation period of 
the refresh timing counter 105 needs to be 16 /jsec. Thus, 
1,024 refreshes per 16.4 msec are achieved. 

It is assumed here that one horizontal 
synchronizing period of an image pickup or display operation 
is 85 £isec. The upper limit of the refresh holding counter 
114 needs to be determined to satisfy: 

z > x/y, (1) 

where x represents the horizontal synchronizing period, y 
represents the pulse generation period of the refresh timing 
counter 105, and z represents the upper limit of the refresh 
holding counter 114. If the upper limit of the refresh 
holding counter 114 is selected to be "16", the maximum 
occupation time period of the DRAM bus 113 becomes 256 ysec. 
Since this time period is sufficiently longer than the 
horizontal synchronizing period, the system can operate 
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stably . 

A CPU access operation will be described below. 
The CPU 102 has an input terminal HOLDRQ and an output 
terminal HOLDAK. If a signal of "1" is inputted to the 
input terminal HOLDRQ, the CPU 102 temporarily stops its 
operation, and outputs a signal of "1" at the output 
terminal HOLDAK to indicate that the CPU 102 has stopped the 
operation. Then, if a signal of "0" is inputted to the 
input terminal HOLDRQ, the CPU 102 outputs a signal of "0" 
at the output terminal HOLDAK and restarts the operation. A 
signal line HOLDRQ of the DRAM bus arbitrating circuit 101 
is connected to the input terminal HOLDRQ of the CPU 102. 
Before a process or a refresh is started, the DRAM bus 
arbitrating circuit 101 outputs a signal of "1" to the input 
terminal HOLDRQ of the CPU 102 and temporarily stops the CPU 
102. If neither the process nor the refresh needs to be 
performed, the DRAM bus arbitrating circuit 101 outputs a 
signal of "0" to the input terminal HOLDRQ and enables the 
CPU 102 to perform the operation. 

The DRAM bus arbitrating circuit 101 which is 
responsible for the above-described control will be 
described below. Fig. 5 is a state transition diagram 
showing the internal state transition of the DRAM bus 
arbitrating circuit 101. The DRAM bus arbitrating circuit 
101 is arranged to start any of a refresh, a CPU access and 
a process in a state "a", and bring them to an end in the 
state "a". 

First of all, the state transition of the DRAM bus 
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arbitrating circuit 101 during the process operation will be 
described below. The DRAM bus arbitrating circuit 101, when 
it is idle, is in the state "a" shown in Fig. 5. If the 
process circuit receives a command and a process request 
signal P_REQ indicative of "1" is outputted to the DRAM bus 
arbitrating circuit 101, the DRAM bus arbitrating circuit 
101 transitions to a state "b". In the state "b", the DRAM 
bus arbitrating circuit 101 outputs a signal of "1" to the 
input terminal HOLDRQ of the CPU 102. When the CPU 102 
receives the signal of "1", the CPU 102 temporarily stops 
the operation and outputs a signal of "1" at the output 
terminal HOLDAK. When the DRAM bus arbitrating circuit 101 
receives the signal of "1", the DRAM bus arbitrating circuit 
101 transitions to a state "c" and outputs a signal P_ACK 
indicative of "1". The process circuit receives this signal 
P_ACK and starts its process processing. When the process 
circuit completes the process processing, a process request 
signal P_REQ indicative of "0" is outputted to the DRAM bus 
arbitrating circuit 101. In response to this signal P_REQ, 
the DRAM bus arbitrating circuit 101 transitions to a state 
"d". In the state "d", the DRAM bus arbitrating circuit 101 
outputs a signal of "0" to the input terminal HOLDRQ of the 
CPU 102. When the CPU 102 receives this signal, the CPU 102 
outputs a signal of "0" at the output terminal HOLDAK and 
restarts the processing. When the DRAM bus arbitrating 
circuit 101 receives this signal, the DRAM bus arbitrating 
circuit 101 transitions to the state "a". 

The state transition of the DRAM bus arbitrating 
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circuit 101 during refresh will be described below. When in 
the state "a", the DRAM bus arbitrating circuit 101 is in an 
idle state. If a refresh request signal R_REQ indicative of 
"1" is out putted to the DRAM bus arbitrating circuit 101 
during the idle state, the DRAM bus arbitrating circuit 101 
receives the refresh request signal R_REQ and transitions to 
a state "e". When in the state "e", the DRAM bus 
arbitrating circuit 101 outputs a signal of "1" to the input 
terminal HOLDRQ of the CPU 102. When the CPU 102 receives 
this signal, the CPU 102 temporarily stops the operation and 
outputs a signal of "1" at the output terminal HOLDAK. When 
the DRAM bus arbitrating circuit 101 receives the signal of 
"1", the DRAM bus arbitrating circuit 101 transitions to a 
state "f" and outputs a signal R_ACK indicative of "1". 

When the refresh signal generating circuit 106 
receives the signal R_ACK indicative of "1", the refresh 
signal generating circuit 106 outputs a refresh pulse for 
one refresh cycle to the DRAM bus 113 and refreshes the DRAM 
103. when one cycle of refresh is completed, the DRAM bus 
arbitrating circuit 101 outputs a signal R_ACK indicative of 
"0" and, at the same time, the count value of the refresh 
holding counter 114 is decremented by one. Thus, the DRAM 
bus arbitrating circuit 101 transitions to a state "g". 
When in the state "g", the DRAM bus arbitrating circuit 101 
outputs a signal of "0" to the input terminal HOLDRQ of the 
CPU 102. In response to this signal, the CPU 102 restarts 
the operation and outputs a signal of "0" at the output 
terminal HOLDAK. Thus, the DRAM bus arbitrating circuit 101 
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transitions to the state "a" and completes the refresh 
operation. At this time, if the count value of the refresh 
holding counter 114 is not "0", it indicates that a signal 
R_REQ indicative of "0" has been outputted from the DRAM bus 
arbitrating circuit 101, so that the above-described refresh 
state transition cycle is again executed. 

The operation of the DRAM bus arbitrating circuit 
101 during CPU access will be described below. When the 
DRAM bus arbitrating circuit 101 is in the state "a", the 
CPU 102 can access the DRAM 103. If the CPU 102 accesses 
the DRAM 103, the DRAM access circuit 104 outputs a signal 
C_REQ indicative of "1" to the DRAM bus arbitrating circuit 
101. When the DRAM bus arbitrating circuit 101 receives the 
signal C_REQ, the DRAM bus arbitrating circuit 101 
transitions to a state "h" . Accordingly, during the CPU 
access, neither a process nor a refresh is activated. When 
the CPU access is completed, the DRAM access circuit 104 
outputs a signal CJREQ indicative of "1" to the DRAM bus 
arbitrating circuit 101. When the DRAM bus arbitrating 
circuit 101 receives the signal C_REQ, the DRAM bus 
arbitrating circuit 101 transitions to the state "a" and 
returns to the idle state. 

The above-described operations are summarized 
below. During the operation of the image pickup circuit 
107, while the image pickup circuit 107 is reading one line 
of image data from a CCD (not shown), the image pickup 
circuit 107 uses the DRAM bus 113 with the highest priority. 
Similarly, during the operation of the image display circuit 
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108, while the image display circuit 108 is outputting one 
line of image data to a monitor (not shown), the image 
display circuit 108 uses the DRAM bus 113 with the highest 
priority. During the blanking periods of an image pickup or 
display operation, refresh is performed with priority during 
the first time period, and the CPU 102 accesses the DRAM 103 
during the remaining time period. 

During the operation of each of the image 
compressing/expanding circuit 109, the vertical addition 
mixing circuit 110 and the image processing circuit 111, 
refresh is performed in preference to their operation. It 
is to be noted that a time period elapses until these 
circuits set the process request signal P_REQ to "1" after 
the refresh, the time period being equivalent to the 
difference between the time period set in the time setting 
register 405 and the time period actually required for the 
refresh. During such time period, the CPU 102 can access 
the DRAM 103. 

The set value of the time setting register 405 for 
each of processes to be executed by the image 
compressing/expanding circuit 109, the vertical addition 
mixing circuit 110 and the image processing circuit 111 will 
be described below. By optimizing the set value of the time 
setting register 405 for each of the circuits 109, 110 and 
110, it is possible to optimize the proportion of the 
operation-enabled time period of each of the circuits 109, 
110 and 110 to the operation -enabled time period of the CPU 
102. 
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In the present embodiment, a set value to be 
written into the time setting register 405 in the vertical 
addition mixing circuit 110 is selected to be smaller than 
those to be written into the respective time setting 
registers 405 of the other circuits 109 and 111. This is 
because, at the time of vertical addition mixing processing, 
the CPU 102 needs only to detect whether vertical addition 
mixing processing has been completed, and the aforesaid set 
value may be a required minimum value. 

However, during image compressing/expanding 
processing, if characters such as "COMPRESSING" are to be 
displayed on a display panel (not shown), the set value for 
the image compressing/expanding circuit 109 needs to be made 
larger than that for the vertical addition mixing circuit 
110 so that the CPU 102 can optimally perform the operation 
of displaying such characters. 

In the case of the image processing circuit 111 as 
well, the set value of the time setting register 405 may be 
adjusted in a similar manner. However, if the set value of 
the time setting register 405 of the image processing 
circuit 111 is made larger, the average speed of the process 
of the image processing circuit 111 becomes slower; whereas 
if the set value is made smaller, the average speed of the 
process becomes faster. In the present embodiment, 
characters such as those formed by the CPU 102 are displayed 
on the display panel during the operation of the image 
compressing/expanding circuit 109, and the CPU 102 
participates in the setting of the processing contents or 
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the like of the image processing circuit 111. Therefore, 
the respective set values of the time setting registers 405 
of the circuits 109, 110 and 111 are made larger in the 
order of the circuit 109, the circuit 111 and the circuit 
110. 

An additional description will be given in 
connection with the image compressing/expanding circuit 109. 
The image display circuit 108 outputs a signal of "0" when 
it is not performing an image display operation or during 
the blanking periods of the image display operation. This 
signal is applied to one input terminal of the OR circuit 
115. The output of the refresh timing counter 105 is 
applied to the other input terminal of the OR circuit 115. 
The output of the OR circuit 115 is connected to an input 
VR_REQ of the image compressing/expanding circuit 109. 

In this arrangement, when the image display 
circuit 108 is not in operation, the image 

compressing/expanding circuit 109 is allowed to operate when 
no refresh is needed; whereas when the image display circuit 

108 is in operation, the image compressing /expanding circuit 

109 is allowed to operate only when no refresh request takes 
place during the blanking periods of an image display 
operation. Accordingly, it is possible to realize time- 
division parallel processing of an image display operation 
and image compressing/expanding processing. If this time- 
division parallel processing is used, it is possible to 
smoothly rewrite an image currently displayed on the screen 
of a monitor with the next different image without stopping 
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the operation of displaying an image on the monitor. As a 
matter of course, when the image display circuit 108 is not 
performing an image display operation, the image 
compressing/expanding circuit 109 operates as a normal 
process circuit of the type B. Such an arrangement is also 
useful for the image processing circuit 111 because the 
arrangement enables the image processing circuit 111 to 
perform processing, such as a variation in the size of an 
image or a rotation of the image on the screen of the 
monitor, while displaying the image. 

According to the present embodiment, in an image 
pickup or display operation, refresh is performed during 
only each blanking period so that there is no possibility 
that an interruption occurs in the image pickup or display 
operation during one line period. In addition, it is 
possible to realize time-division processing of both a 
process which reguires a comparatively long processing time 
period, such as image compression/expansion, vertical 
addition mixing or image processing, and the access of the 
CPU 102 to the DRAM 103. This feature enables, for example, 
the function of blinking a display such as "COMPRESSING" on 
the monitor during the compression of an image. 
Furthermore, since the DRAM 103 is refreshed by an optimum 
number of times, power consumption due to refresh can be 
minimized and the interruption of the aforesaid processing 
can be minimized to reduce the entire processing time. 
Accordingly, it is possible to provide a digital electronic 
camera of good operability. 
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In addition, the image display circuit 108 and the 
image pickup circuit 107 as well as the image display 
circuit 108 and the image processing circuit 111 can be 
operated apparently at the same time in a time-division 
manner, whereby it is possible to realize processing such as 
the rewriting of data on the monitor or the rotation of an 
image on the monitor during the display of the image on the 
monitor . 

Furthermore, since no refresh timing is obtained 
from the image pickup circuit 107, the supply of power to 
the image pickup circuit 107 can be stopped if no image 
pickup operation needs to be performed, whereby it is 
possible to realize a further decrease in power consumption. 

As is apparent from the foregoing description, 
according to the present embodiment, since various kinds of 
image processing are divided into first processing which can 
be performed with priority and second processing over which 
refresh takes priority, it is possible to appropriately 
effect the various kinds of image processing by using a 
memory which needs refreshing, such as a DRAM, while 
reliably refreshing the memory. 

Since the first processing is assigned the 
processing of storing picked-up image data and the 
processing of displaying the stored image data, it is 
possible to prevent an interruption in an image pickup or 
display operation. 

Since the second processing is assigned processing 
which requires a comparatively long processing time period, 
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for example , compression/expansion of image data, vertical 
addition and size change, such processing can be performed 
in a time-division manner, whereby it is possible to provide 
display during the processing. In addition, since refresh 
is performed by an optimum number of times, power 
consumption can be reduced. 

In addition, even in the first processing, it is 
possible to reliably effect refresh during the blanking 
periods of picked-up image data or display image data. 

In addition, since it is possible to effect 
refresh during blanking periods or respond to a refresh 
request while performing compressing/expanding processing or 
image processing, it is possible to effect rewriting or 
processing of image data on the monitor while displaying an 
image on the monitor. 

In addition, access to a memory by control means 
such as a CPU can be effected at appropriate timing during 
each processing. 

in addition, since refresh is performed with a 
counter whose count value increments by one for a 
predetermined time period and decrements by one for each 
refresh, it is possible to reliably effect refresh within a 
required time period. 

In other words, in accordance with the present 
embodiment, since an image pickup or display operation is 
performed without interruption, an image pickup operation 
can be completed in a shortest image pickup time period and 
a high-quality display screen free of noise can be achieved. 
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Since both a process which requires a 
comparatively long processing time period, such as image 
compress ion /expansion, vertical addition mixing or image 
processing, and access to a DRAM by a CPU can be executed by 
time-division processing, it is possible to realize the 
function of blinking a display such as "COMPRESSING" on the 
monitor during, for example, the compression of an image. 

Power consumption due to refresh can be minimized, 
and the supply of power to an image pickup circuit can be 
stopped, whereby it is possible to realize a further 
decrease in power consumption. 

In addition, it is possible to realize processing 
such as the rewriting of data on the monitor or the rotation 
of an image on the monitor during the display of the image 
on the monitor. 

As is apparent from the above description, it is 
possible to provide an apparatus such as a digital 
electronic camera of good operability. 

Furthermore, since the operating time of the CPU 
is optimally set for each image processing circuit, the 
average processing time period of each image processing 
circuit can be reduced, and it is possible to optimize the 
processing time period of each image processing circuit 
according to whether the CPU needs to create display data. 
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CLAIMS 



1- An image processing apparatus comprising: 
storage means for storing image data; 
refresh means for performing a refresh operation 

for refreshing said storage means within a predetermined 

time period; 

first and second process means for respectively 
performing their predetermined processes by using said 
storage means ; and 

arbitrating means for performing arbitration to 
cause said first process means to perform the predetermined 
process as a priority over the refresh operation and to 
cause said second process means to perform the predetermined 
process while allowing the refresh operation to be performed 
as a priority over the predetermined process. 

2. An image processing apparatus according to claim 
1, wherein said first process means includes at least one of 
image pickup means for storing picked-up image data in said 
storage means and display means for displaying the image 
date stored in said storage means. 

3. An image processing apparatus according to claim 
1, wherein said second process means includes at least one 
of compressing/expanding means for compressing/expanding the 
image data, adding means for adding together vertical 
adjacent data of the image data, and processing means for 
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processing the image data. 

4. An image processing apparatus according to claim 
1, wherein said arbitrating means causes said refresh means 
to perform the refresh operation during a blanking period of 
the picked-up image data or the image data to be displayed. 

5. An image processing apparatus according to claim 
1, wherein said first process means includes display means 
for displaying the image date stored in said storage means, 
and, while said second process means is operating, allows 
the refresh operation to be performed on the basis of a 
refresh request from said refresh means during a blanking 
period of the image data to be displayed. 

6. An image processing apparatus according to claim 
1, further comprising control means for controlling another 
operation of said image processing apparatus, a control 
program for said control means being stored in said storage 
means, said control means accessing said storage means 
during an idle time period other than a time period during 
which any of said refresh means and said first and second 
process means accesses said storage means. 

7. An image processing apparatus according to claim 
1, wherein said arbitrating means includes counting means 
for counting up to a predetermined number, said counting 
means being arranged to increment a count value by one 
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within the predetermined time period and, each time the 
refresh operation is performed once, decrement the count 
value by one, said refresh means performing the refresh 
operation when the count value is not less than 1. 

8. A computer-readable recording medium in which is 
stored a program for executing: 

a storage process for storing image data in 
storage means; 

a refresh process for performing a refresh 
operation for refreshing the storage means within a 
predetermined time period; 

a first process for performing a predetermined 
process as a priority over the refresh operation by using 
the storage means ; and 

a second process for performing a predetermined 
process while allowing the refresh operation to be performed 
as a priority over the predetermined process. 

9. A recording medium according to claim 8, wherein 
said first process includes at least one of an image pickup 
process for storing picked-up image data in the storage 
means and a display process for displaying the image date 
stored in the storage means. 

10. A recording medium according to claim 9, wherein 
said second process includes at least one of a 
compressing /expanding for compressing /expanding the image 
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data, an adding process for adding together vertical 
adjacent data of the image data, and a processing process 
for processing the image data. 

11. A recording medium according to claim 9, wherein 
said refresh process is performed during a blanking period 
of the picked-up image data or the image data to be 
displayed. 

12. A recording medium according to claim 8, wherein 
said first process includes a display process for displaying 
the image date stored in the storage means, and, while said 
second process is operating, allows the refresh operation to 
be performed on the basis of a refresh request during a 
blanking period of the image data to be displayed. 

13. An image processing apparatus comprising: 
storage means for storing image data; 

refresh means for refreshing said storage means; 

a plurality of process means each of which 
performs writing of image data into said storage means or 
reading of image data from said storage means; and 

arbitrating means for arbitrating operations of 
said plurality of process means and an operation of said 
refresh means. 

14. An image processing apparatus according to claim 
13, wherein said arbitrating means specifies whether to 
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synchronize the operation of said refresh means with image 
data, according to the kind of process executed by each of 
said plurality of process means. 

15. An image processing apparatus according to claim 

14, wherein said arbitrating means synchronizes the 
operation of said refresh means with a horizontal 
synchronizing signal of the image data while particular 
process means among said plurality of process means is 
operating, said particular process means being arranged to 
perform an image input process or an image output process. 

16. An image processing apparatus according to claim 

15, wherein said arbitrating means sets a period of the 
operation of said refresh means to a predetermined period 
determined on the basis of a characteristic of said storage 
means while particular process means among said plurality of 
process means is operating, said particular process means 
being arranged to perform an image processing process or an 
image compressing process. 

17. An image processing apparatus according to claim 
13, further comprising control means for exerting system 
control over said image processing apparatus by using said 
storage means, said control means being operated during a 
transition from a state in which said refresh means operates 
to a state in which said plurality of process means operate. 
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18. An image processing apparatus comprising: 
storage means for storing image data; 
refresh means for refreshing said storage means; 
process means for writing image data into said 

storage means or reading image data from said storage means; 

and 

control means for controlling another operation of 
said image processing apparatus by using said storage means, 

wherein said control means is operated during a 
transition from a state in which said refresh means operates 
to a state in which said process means operates. 



19. An image processing apparatus according to claim 
18, wherein said process means is arranged to perform an 
image input process or an image output process, and said 
refresh means is arranged to operate during a horizontal 
blanking period of image data. 



20. An image processing apparatus comprising: 
storage means for storing image data; 
refresh means for refreshing said storage means; 

and 

a plurality of process means each of which 
performs writing of image data into said storage means or 
reading of image data from said storage means, 

wherein a period of an operation of said refresh 
means is switched according to the kind of process executed 
by process means which is operating, among said plurality of 
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process means. 



21. An image processing apparatus according to claim 

20, wherein the operation of said refresh means is 
synchronized with a horizontal synchronizing signal of the 
image data if the process executed by said process means is 
an image input process or an image output process . 

22. An image processing apparatus according to claim 

21, wherein a period of the operation of said refresh means 
is set to a predetermined period determined on the basis of 
a characteristic of said storage means if the process 
executed by said process means is an image processing 
process or an image compressing process. 

23. An image processing apparatus comprising: 
storage means for storing image data; 

refresh means for refreshing said storage means; 

and 

process means for performing writing of image data 
into said storage means or reading of image data from said 
storage means, 

wherein said refresh means is operated during only 
a blanking period selected from among blanking periods of a 
synchronizing signal of said image data. 

24 . An image processing apparatus according to claim 
23, wherein an average refresh period of said refresh means 
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is set to a period determined on the basis of a 
characteristic of said storage means. 

25. An image processing apparatus comprising: 

(a) storage means for storing image data; 

(b) refresh means for refreshing said storage means; 

(c) a plurality of process means each of which 
performs writing of image data into said storage means or 
reading of image data from said storage means; 

(d) control means for controlling another operation by 
using said storage means; and 

(e) arbitrating means for arbitrating access to said 
storage means so that a period during which to operate said 
control means and a period during which to operate process 
means which is operating, among said plurality of process 
means, is relatively switched therebetween, according to the 
kind of process means which is operating. 

26. An apparatus according to claim 25, wherein said 
arbitrating means causes said refresh means to perform a 
predetermined operation, and disables processes of said 
plurality of process means for a predetermined time period 
and causes said control means to operate until the end of 
the predetermined time period after the completion of the 
predetermined operation of said refresh means. 

27. An apparatus according to claim 26, wherein the 
predetermined time period is determined according to each of 
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said plurality of process means. 

28. An apparatus according to claim 27, wherein each 
of said plurality of process means has a register for 
holding a value which corresponds to the predetermined time 
period 7 said control means being capable of rewriting said 
register . 

29. An apparatus according to claim 28, wherein said 
plurality of process means include first process means for 
reading image data stored in said storage means, in order to 
display the image data, and a plurality of second process 
means for varying said image data into different forms, 
respectively, said arbitrating means being arranged to 
relatively switch, according to the kind of second process 
means which is operating, among said plurality of second 
process means, a period during which to operate said control 
means and a period during which to operate said second 
process means which is operating, during a blanking period 
of image data displayed by said first process means . 

30. An image processing method comprising steps of: 

(a) performing a refresh process on storage means 
which stores image data; 

(b) performing a first process by using the storage 
means ; 

(c) performing a second process by using the storage 
means ; and 
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(d) arbitrating access to the storage means in such a 
manner that the first process takes priority over the 
refresh process and the refresh process takes priority over 
the second process . 

31. An image processing method comprising steps of: 

(a) performing a refresh process on storage means 
which stores image data; 

(b) processing image data by performing writing of the 
image data into the storage means or reading of the image 
data from the storage means; and 

(c) specifying whether to synchronize the refresh 
process with an image, according to a process content of 
said image processing step. 

32 . A computer-readable recording medium in which is 
stored a program for executing: 

a step of performing a refresh process on storage 
means which stores image data; 

a step of processing image data by performing 
writing of the image data into the storage means or reading 
of the image data from the storage means; and 

a step of specifying whether to synchronize the 
refresh process with an image, according to a process 
content of said image processing step. 

33. An image processing method comprising steps of: 
(a) performing a refresh process on storage means 
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which stores image data; 

(b) processing image data by performing writing of the 
image data into the storage means or reading of the image 
data from the storage means; 

(c) controlling another operation by using the storage 
means ; and 

(d) executing said control step during switching from 
said refresh process step to said image data processing 
step. 

34. A computer-readable recording medium in which is 
stored a program for executing: 

a step of performing a refresh process on storage 
means which stores image data; 

a step of processing image data by performing 
writing of the image data into the storage means or reading 
of the image data from the storage means; 

a step of controlling another operation by using 
the storage means; and 

a step of executing said control step during 
switching from said refresh process step to said image data 
processing step. 

35. An image processing method comprising steps of: 

(a) performing a refresh process on storage means 
which stores image data; 

(b) applying a first process to image data by 
performing writing of the image data into the storage means 
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or reading of the image data from the storage means; 

(c) applying a second process to image data by 
performing writing of the image data into the storage means 
or reading of the image data from the storage means; and 

(d) switching a period of the refresh process between 
when the first process is performed and when the second 
process is performed. 

36. A computer-readable recording medium in which is 
stored a program for executing: 

a step of performing a refresh process on storage 
means which stores image data; 

a step of applying a first process to image data 
by performing writing of the image data into the storage 
means or reading of the image data from the storage means; 

a step of applying a second process to image data 
by performing writing of the image data into the storage 
means or reading of the image data from the storage means; 
and 

a step of switching a period of the refresh 
process between when the first process is performed and when 
the second process is performed. 

37. An image processing method comprising steps of: 

(a) performing a refresh process on storage means 
which stores image data; 

(b) applying a process to image data by performing 
writing of the image data into the storage means or reading 
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of the image data from the storage means; and 

(c) performing the refresh process during only a 
blanking period selected from among blanking periods of an 
image signal indicative of the image data. 

38. A computer-readable recording medium in which is 
stored a program for executing: 

a step of performing a refresh process on storage 
means which stores image data; 

a step of applying a process to image data by 
performing writing of the image data into the storage means 
or reading of the image data from the storage means; and 

a step of performing the refresh process during 
only a blanking period selected from among blanking periods 
of an image signal indicative of the image data. 

39. An image processing method comprising steps of: 

(a) performing a refresh process on storage means 
which stores image data; 

(b) applying a first process to image data by 
performing writing of the image data into the storage means 
or reading of the image data from the storage means; 

(c) applying a second process different from the first 
process to image data by performing writing of the image 
data into the storage means or reading of the image data 
from the storage means; 

(d) controlling another operation of an image 
processing apparatus by the storage means; and 
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(e) relatively switching a time period during which to 
control said other operation and a time period during which 
to execute the first or second process, according to whether 
the first process or the second process is being performed. 

40. A computer-readable recording medium in which is 
stored a program for executing: 

a step of performing a refresh process on storage 
means which stores image data; 

a step of applying a first process to image data 
by performing writing of the image data into the storage 
means or reading of the image data from the storage means; 

a step of applying a second process different from 
the first process to image data by performing writing of the 
image data into the storage means or reading of the image 
data from the storage means; 

a step of controlling another operation of an 
image processing apparatus by the storage means; and 

a step of relatively switching a time period 
during which to control said other operation and a time 
period during which to execute the first or second process, 
according to whether the first process or the second process 
is being performed. 
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ABSTRACT OF THE DISCLOSURE 

In order to reliably execute various kinds of 
image processing while reliably performing a refresh 
operation, during the execution of a refresh process for a 
storage element which stores image data, a first process 
using the storage element and a second process using the 
storage element, access to the storage element is arbitrated 
so that the first process takes priority over the refresh 
process and the refresh process takes priority over the 
second process. In addition, in order to reduce the refresh 
rate of the storage element to the minimum necessary rate to 
reduce power consumption and to enhance the speeds of other 
processes, during the execution of a refresh process for 
refreshing the storage element which stores image data and a 
process for processing the image data by performing writing 
of the image data into the storage element or reading of the 
image data from the storage element, the manner of the 
refresh process is switched according to the contents of the 
image data processing process. 
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